Parallelism-Aware Batch Scheduling: Paving the Way to High-Performance and Fair Memory Controllers

In modern processors, the DRAM system is shared among concurrently-executing threads. Memory requests from a thread can delay requests from other threads by causing bank/bus/rowbuffer conflicts. Conventional DRAM controllers are unaware of inter-thread interference, which causes two problems. First, some threads are unfairly penalized and denied DRAM service for long time periods. Second, as we show in our ISCA-35 paper, each thread’s memory-level parallelism can be destroyed. A thread’s outstanding requests that would have been serviced in parallel can effectively become serialized, exposing the latency of each request. As a result, both single-thread performance and system performance/fairness degrade. Our ISCA-35 paper proposes parallelism-aware batch scheduling (PAR-BS), a new approach to designing a shared DRAM controller. PAR-BS is based on two new basic building blocks which collectively reduce inter-thread interference in DRAM, ensure fairness, and preserve the memory-level parallelism of each thread. As a result, PAR-BS reduces the memory-related stalltime experienced by the threads. In addition, PAR-BS provides fairness, avoids starvation of any thread, and seamlessly incorporates support for system-level thread priorities. Our evaluations show that PAR-BS significantly improves both fairness and system performance compared to four previous DRAM controllers across a wide variety of workloads and systems.